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AN  EXPONENTIAL  AUTOREGRESSIVE-MOVING  AVERAGE  PROCESS  EARMA(p,q) 
DEFINITION  AND  CORRELATIONAL  PROPERTIES 


A.  J.  Lawrance 
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Birmingham,  England 

and 

P.  A.  W.  Lewis 
Naval  Postgraduate  School 
Monterey,  California 


ABSTRACT 

A  new  model  for  pth-order  autoregressive  processes  with 
exponential  marginal  distributions  EAR(p)  is  developed  and  an 
earlier  model  for  first  order  moving  average  exponential  processes 
is  extended  to  qth-order,  giving  an  EMA(q)  process.   The  correla- 
tion structure  of  both  processes  are  obtained  separately.   A 
mixed  process,  EARMA(p,q),  incorporating  aspects  of  both  EAR(p) 
and  EMA(q)  correlation  structures  is  then  developed.   The 
EARMA(p,q)  process  is  an  analog  of  the  standard  ARMA(p,q)  time 
series  models  for  Gaussian  processes  and  is  generated  from  a 
single  sequence  of  independent  and  identically  distribution 
exponential  variables . 


1.   INTRODUCTION 

The  first-order  autoregressive  sequence,  EAR(l),  was 
introduced  by  Gaver  and  Lewis  (1975-1978)  with  the  primary  aim 
of  generalizing  the  Poisson  model  for  point  processes  to  one  in 
which  the  intervals  between  events  were  correlated  but  still  had, 
marginally,  exponentially  distributed  intervals.   The  EAR(l) 
sequence  is  a  simple  random  linear  combination  of  independent 
exponential  random  variables  whose  properties  are  relatively 
simple  to  derive.   This  is  in  contrast  to  previous  attempts  to 
generalize  the  Poisson  process  via  Markov  dependence  which  led 
to  intractable  models  (see  e.g.  Wold,  1948  and  Cox,  1955). 

In  Lawrance  and  Lewis  (1977)  another  sequence  of  dependent 
exponential  random  variables  was  introduced.   This  sequence, 
called  EMA(l),  was  again  a  random  linear  combination  of  independent 
exponential  random  variables,  but  had  the  dependency  properties 
of  a  first  order  moving  average  process .   The  first-order  moving 
average  and  autoregressive  processes  were  combined  by  Jacobs 
and  Lewis  (1977)  to  form  the  EARMA(1,1)  sequence.   Jacobs  and 
Lewis  (1977)  gave  stationary  initial  conditions  and  mixing 
properties  of  the  sequences,  these  results  applying  to  the  EAR(l) 
and  EMA(l)  processes  as  special  cases. 

In  the  present  paper  we  extend  these  results  and  describe 
a  mixed  pth-order  autoregressive,  qth-order  moving  average  process 
with  exponential  marginal  distributions  which  we  denote  as 
EARMA(p,q) .   The  process  is  again  a  random  linear  combination 


of  independent  exponential  variables,  and  as  such  is  simple  to 
generate  on  a  computer;  it  will  thus  be  useful,  for  example,  in 
simulation  studies  of  queues  with  correlated  interarrival  times 
or  service  times  (see  Jacobs,  1978).   The  process  is  not  unique, 
but  its  correlation  sequence   {p  }   does  satisfy  equations  like 
the  Yule-Walker  equations  which  arise  in  the  study  of  linear 
processes  (see  e.g.  Feller,  1966,  or  Box  and  Jenkins  1970). 

It  is  perhaps  well  to  reiterate  the  essential  difference 
between  the  EARMA(p,q)  process  and  the  ARMA(p,q)  process;  this 
is  that  the  EARMA(p,q)  process  is  defined  to  have  an  exponential 
marginal  distribution.  It  is  not  known  how  one  would  pick  the 
error  sequence  in  the  ARMA(p,q)  sequence  to  make  it  have,  even 
approximately,  marginal  exponential  distributions.   In  fact  the 
marginal  distributions  would  tend  to  be  approximately  normally 
distributed  for  most  error  sequences  (Mallows,  1967);  the  catch 
in  this  result  is  that  the  distribution  of  the  error  sequence  be 
independent  of  the  parameters  of  the  moving  average  and  auto- 
regression.   This  is  not  so  for  the  EARMA(p,q)  process. 

This  paper  will  be  limited  to  definitions  and  to  description 
of  the  correlational  properties  of  the  EARMA(p,q)  process.   In 
Section  2  the  EAR(p)  model  is  introduced  and  an  explicit  solution 
for  the  required  error  process  for  autoregression  of  order  2  is 
given.   In  Section  3  we  describe  the  extensions  of  the  EMA(l) 
model  to  the  EMA(g)  model;  these  are  relatively  straightforward 
and  are  indicated  in  Lawrance  and  Lewis  (1977)  .   The  general 


EARMA(p,q)  model  is  introduced  in  Section  4,  and  specific  results 
are  obtained  for  the  EARMA(1,1),  EARMA(2,1)  and  EARMA(1,2)  models  in 
Section  5. 

In  deriving  correlational  properties  it  is  assumed  that 
the  EARMA(p,q)  process  is  stationary.   The  question  of  stationarity , 
stationary  initial  conditions  and  mixing  properties  will  be  con- 
sidered elsewhere,  as  will  be  questions  of  distributions  of  sums 
of  the  dependent  variables  and  spectra  of  point  processes  with 
EARMA(p,q)  interval  structure.   There  are  also  open  questions  of 
estimation  of  parameters  and  fitting  to  data. 

We  note  too  that  there  is  a  mild  degeneracy  to  the  EAR(p) 
process  in  that  one  obtains  runs  in  which  the  variables  are 
scaled  versions  of  the  previous  variables.   This  disappears  when 
the  moving  average  component  is  introduced.   Another  drawback  is 
that,  unlike  the  ARMA(p,q)  model,  only  positive  valued  serial 
correlations  can  be  obtained  from  the  EARMA(p,q)  model  and  while 
much  data  appears  to  be  of  this  type  (see  e.g.  Lewis  and  Shedler, 
1977),  it  is  a  drawback.   This  can  be  overcome  by  considering 
antithetic  processes  but  this  aspect  of  the  model  is  beyond  the 
scope  of  the  present  paper. 


2.   THE  EXPONENTIAL  AUTOREGRESSIVE  EAR (p)  MODEL 
2a .   Definition . 

The  standard  linear,  first-order  autoregressive  model 
for  a  stationary  sequence  of  random  variables   {X.}   is  defined 
by  the  equation 


X.  =  pX.  ,  +  e.     ,  i  =  0f+l,+2,  ...  ,        (2.1) 

l      l-l     i  —  —       ' 


where   p   is  a  constant  which  is  less  than  1  in  absolute  value 
and  {e.}  is  a  sequence  of  independent  and  identically  distributed 
random  variables.   Gaver  and  Lewis  (1975-1978)  showed  that  if 
the   {X.}   sequence  were  to  have  an  exponential  marginal  distri- 
bution with  parameter   A ,  then  the  parameter  p   should  be  greater 

than  or  equal  to  zero  and  less  than  one,  and  e.       should  be  zero 

l 

with  probability   p   and  an  exponential (A )  random  variable,  E., 
with  probability   1-p.   Thus 


Xi  =  pXi-l  +  ei  i  =  °'±1't2>       "  , 


pXi_1        w.p.    p  , 
pXi_1  +  E^^    w.p.   1-p, 


i  =  0,+l,+2,  . . .  ,  (2.2) 


where   {E.}   is  an  i.i.d.  sequence  of  exponential (A )  random 
variables.   Note  that  for  this  EAR(l)  model  the  distribution  of 
the   £.   depends  on   p,  the  multiplicative  weight  of   X._,. 


This  violates  an  assumption  which  is  implicit  in  many  applica- 
tions of  (2.1),  the  so-called  AR(1)  model.   In  particular  standard 
results  showing  that  the   {X.}   sequence  becomes  a  normal  process 
as   p  -*■  1   for  any   {e.}   sequence  are  invalid;  in  the  EAR(l) 
process  the   X. 's  always  have,  by  construction,  an  exponential ( - ) 
marginal  distribution. 

Generalization  of  the  usual  higher-order  autoregressive 
models   AR(p)   based  on  extensions  of  (2.1)  to  higher  order 
autoregressive  exponential  processes  is  difficult.   This  is 
because  it  is  not  possible  to  solve  the  defining  equation  for  the 
distribution  of  the   e. '  s,  if  it  exists.   We  present  here  a 
different  type  of  p-th  order  autoregressive  models  with 
exponential  marginal  distributions.   They  share  with  the  AR(p) 
models  the  same  correlation  structure,  are  p-th  order  Markov 
processes,  and  are  (autoregressively)  functions  of  at  least  one 
of  the  previous   p   variables. 

The  second-order  model,  EAR (2),  takes  the  form 


a, X .  ,     w .p .   1-a 
1  l-l       ^ 


2 


X.  =  j  (  +  e.  2.3 

a„X .  _.     w  .p .    a~ 
2  i-2       ^  2 

where   a,   and   a„   are  constants   (0  <  a,,  a„  <  1)   and  we  show 
later  that  the  distribution  of  the   e .   is  uniquely  determined 
by  the  requirement  that  the   X. 's   have  exponential  marginal 
distributions.   The  second-order  autoregressive  nature  of  the 

model  is  evident;  X.   is  always  a  function  of  one  of 

l  2 


the 


previous  two  values   X.  ,   and   X.  _.   This  is  in  contrast 
^  l-l         1-2 


to  the  AR(2)  model  in  which   X.   is  a  function  of  a  linear  com- 


bination of   X 


i-1 


and   X . 

i-2 


The  third-order  model  is  given  by 


a  ,  X  .  , 
1  l-l 


X.  =  {  a0X.  0 
l    I   2  i-2 


06  <->  X  •   -> 

3  i-3 


w.p 


1  -  a, 


w.p.   a2 (1-a3)  [   +  £i  • 
w.p.   ct2»3 


(2.4) 


The  p-th  order  model  is  similarly  constructed  and  may  be 


written 


fVi-i 


x.  = 

i 


w.p .    a. 


a2Xi-2     W-P'    a2 


a  X.       w.p.    a 
P  i-P  P 


+   £ 


(2.5) 


where 


and 


az   =   n   a.  (l-aul)  , 


a^j^  =  d-ot2)  , 


I   =  2 ,  .  .  . ,p-l 


a   =   II   a  .  . 
P    j-2   ^ 


(2.6) 


The  mixing  probabilities  and  the  weights  on  the  auto- 
regressed  variables  are  to  some  extent  a  matter  of  choice  (other 
parametrizations  are  clearly  possible)  and  we  have  been  guided 
by  two  considerations;  having  a  minimum  number  of  parameters, 


preferably  the  same  number  of  parameters  as  the  order  of  the 

autoregression ,  and  by  the  need  for  the  autoregression  in  the  EAR(p) 

model  to  reduce  in  order  by  one  when  the  last  coefficient,   a  , 

P 

is  set  to  zero.   In  the  present  parametrization  this  implies  the 
weak  restriction  that  it  is  not  possible  to  suppress  intermediate 
autoregressions ,  i.e.  dependence  of   X.   on   X.     in  a  EARMA ( 2 ) 
model . 

With  regard  to  the  question  of  parametrization,  one  could 
in  (2.3)  replace  the  probabilities   a,=  (1-a  )   and   a   =  a„   by   (1-p 
and   p   and  there  is  then  no  need  for  the  weights   a,   and   ot- 
to be  less  than  or  equal  to  one.   But  if  they  are  not,  the  process 
will  not  reach  equilibrium  unless   p   is  suitably  chosen,  i.e. 
be  stable.   Again,  even  if  the  process  is  stable  it  is  not  clear 
yet  that  the  additional  parameter  adds  any  generality  to  the 
process.   We  consequently  consider  only  the  parametrizations 
given  in  (2.6). 


2b.   The  error  sequence   {e.} 
a 3_ 

We  now  Obtain  the  distribution  of  the  i.i.d.   {c.} 

l 

sequence  which  will  ensure  that  the   {X.}   sequence  in  the  EAR(2) 

model  has  an  exponential  marginal  distribution.   Let  $       (s) 

x  • 

i 

and   $   (s)   be  the  Laplace-Stielt jes  transforms  of  the  marginal 

i 

distributions  of  the   X.'s  and  the   e.'s: 

i  l 

-X.s  -f;  .  s 

$   (s)  =  E(e   X  )  ;       <J>   (s)  =  E(e  1    )     .        (2.7) 

X.  4 

l 

7 


Then  from  Equation  (2.3)  we  have 

4)   (s)  =  [(l-a2)  $     (c^s)  +  a2(})x    (a2s)]  4>  (s)  ,      (2.8) 

X.  i-1  i-2 

l 

where  we  have  used  the  fact  that  expectation  of  the  mixture  of 

two  dependent  random  variables  is  the  mixture  of  the  expectations 

of  the  marginal  random  variables,  here   X.  ,   and   X.  „.   Thus 
^  l-l         i-2 

we  avoid  the  joint  Laplace-Stietl jes  transform  which  comes  in 
when  one  tries  to  solve  the  usual  linear  AR(2)  equations  to 
obtain  an  exponential  process.   Assuming  marginal  stationarity 
for  the  process,  we  have 


X 


(s)    =    [(l-a2)     <|)     (a1s)    +    a2<J>x(a2s)]     (J>     (s)     .  (2.9) 


To  show  that  such  an  error  sequence   { e  .  }   exists  we  solve  (2.9) 

directly  and  invert  the  transform.   We  have  for  the  EAR (2)  model, 

using  the  key  requirement  that  the  marginal  distribution  of  the 

X.'s  be  exponential (A) ,  and  thus   $v(s)  =  A/(A+s),  that 
1  x 


*x(s) 

'e(s)    =     (l-a2)     (J)x(a1s)    +    a2<J>x(a2s)  (2.10) 


( A+a, s) ( A+a    s) 
(A+s) [ (l-a2) (A+a2s)    +    a2(A+«1s)]  (2.11) 


Then  by  a  partial  fraction  expansion 


(S)      =     TT„      +     7Tn      t4~     +     TT, 


ve  0  1    A+s  2    A+Ss 

where   S  =  (1+a. -a?) a2  .   Using  the  fact  that   a,   and   a    are 
probabilities,  it  is  easily  verified  that   tt-,  tt.  ,  and   tt2   are 
positive,  and  since  their  sum  is  equal  to  1,  we  have 
0  <_  7Tn,  tt.  ,  7T_  £  1.   Thus  e      is  a  convex  mixture  of  a  discrete 
component  and  two  exponentials,  and  thus  has  a  proper  distribution, 
This  distribution  is  also  unique,  by  the  unicity  theorem  for 
Laplace-Stielt jes  transforms.   The  complete  specification  of 
e.   is,  for   i  =  0,  +  1,  +_  2'  ... 


0 


w.p .   a, /{ 1+a, -a2] , 


=  <  Ei     w.p.   (l-o^)  (l-a2)/[l-a2  (1+o^-a  )  ]  ,        (2.12) 
Ei/S   w.p.   (l-a2)  (a;L-a2)2/[  (l  +  a;L-a2)  (1-S)  ]  , 


where   {E.}   is  again  an  i.i.d.  sequence  of  exponential ( A ) 
random'  variables .   It  is  obvious  from  (2.12)  that  the  mean  and 
variance  of   e.   depend   on   a    and   a  ,  the  multiplicative 
weights  of   X  ■  -,       and   X_2   respectively,  as  well  as  on   A. 

As  in  the  EAR(l)  model  there  is  a  non-zero  probability 
of   e.   being  zero;  otherwise  it  is   E.   or  a  scaled  version  of 
E. .   The  higher  order  models  can  in  principle  be  similarly 
treated,  although  above  the  third  order  there  will  be  difficulty 
with  the  partial  fraction  expansion. 


2c.   Correlation  structure 

The  correlation  structure  of  the  stationary  EAR(p) 
models  can  be  obtained  by  the  usual  device  of  multiplying  the 
defining  equations  for   X.   by   X.   ,  for   r  =  1,2,...,  and 
taking  expectations.   What  results  are  difference  equations 
which  are  entirely  analogous  to  the  Yule-Walker  type  equations 
obtained  for  the  standard  AR(p)  model. 

Thus  taking  the  EAR (2)  case  as  a  typical  example,  we 
have  from  (2.3)  that 


E(X.X.   )  =  (l-a0) [a,E(X.  ,X.   )  +  E(X.   )  E(e.)] 
l  l-r         2    1    l-l  l-r        l-r      i 


+  a  [a  E(X._„X._  )  +  E(X._  )  E(e.)]  .     (2.13 


-1  -2 

Using  the  fact  that   E(X.)  =  A   ,  var (X. )  =  A   ,  because  the 

process  has  an  exponential  marginal  distribution,  and  from  (2.3 
that 


E(e)  =  (l-a0) (l-a,+a  )  E(X)  ,  (2.14) 

2      1   2 


we  obtain  for  the  correlations   p   =  Corr(X.,X.   )   the  equation, 

Kr  l   l-r         ^ 

2 

Pr  =  a1(l-a2)pr_1  +  a2pr_2     (r  >_  1)        (2.15) 


with   p   =   p_    and   pfi  =  1.   For  the  general  EAR(p)  process 
there  is  the  corrasponding  equation 


10 


p   =  a,a,p  -,    +    a0a„pv  0  +  •••  +  a  a  p       (r  >  1).    (2.16) 
hr     1  1  r-1     2  2  r-2  p  p^r-p       — 

Equation  (2.15)  is  a  system  of  second  order  difference 
equations  from  which  we  can  obtain  the  following  results. 

(i)   The  solution  of  the  difference  equation  (2.15)  is  (see, 
for  example,  Box  and  Jenkins,  1970,  pp.  58-59) 


Pr  =  y^l    +  y2*r2  (r  >  1)  (2.17 


Z1(1"Z2)Z1  "  22(1-zl)z2 
(z-j-z  )  (1+z, z  ) 


(2.18) 


where   z,   and   z    are  reciprocals  of  the  roots  of  the 
characteristic  equation 

1  -  a1 (l-a2)B  -  a^B2  =  0  , 

2       2      2 

and  the  roots  are  real  since   a,  (l-a9)   +  4a?  >  0  .   Also 

0  <_  z-  <  z,  <  1.   An  implication  of  these  results  is  that 
the  serial  correlations  are  positive  and  eventually  decay 
geometrically,  i.e.  like   Y-jzf.   We  have  assumed  that   a2  >  0; 
otherwise  we  have  the  EAR(l)  model, 
(ii)   The  correlations   p,   and   p    can  be  uniquely  defined  in 
terms  of  the  parameters   a,   and  a    ,  and  vice  versa;  this 
follows  from  (2.15)  for   r  =  1   and   r  =  2.   We  have 
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al  2 

pi  =  rr^r  ;  p2  =  a1d-a2)p1  +  a2  ;  (2.19) 


and 


1/2 


p2"pl 


a2=\ T~l  '       ai       +     ~      p"-  (2.20) 

1-P-L 


if   a   ^  0.   If   a„  =  0   the  model  reduces  to  the  EAR(l) 
model  of  Gaver  and  Lewis  (1975-1978),  and   p,  =  a  . 
Equation  (2.20)  may  be  used  to  obtain  Yule-Walker  estimates 
of   a    and  a        from  estimates  of  the  first  two  serial 
correlations . 


2 
(iii)   If   a„  ^  0   then,  unlike  the  EAR(l)  case  in  which   p_  =  p., 

2 

we  have   p_  >  p, .   This  can  be  seen  from  (2.19) ,  which  can 

be  written  as   p2  =  p^  +  u2(1_p2}  ^  p2^   Note  tQQ 
that  there  are  values  of   a,   and   a_   for  which   p   >  p,  ; 
thus  the  additional  degree  of  autoregression  produces,  at 
least  for  the  first  two  serial  correlations,  a  broader 
correlation  structure  than  is  possible  with  the  EAR(l) 
process   (a„  =  0) . 


(iv)    One  way  to  measure  the  amount  of  correlation  which  is  intro- 
duced into  the  sequence   {E^}   by  the  autoregression  is  by 
an  index  of  dispersion  (Cox  and  Lewis,  1966,  p.  71).   This 
is  just  the  limiting  value  of  the  variance  of  the  sum  of 
k   adjacent  to   X. 's,  standardized  by  its  value  for  independent 

exponential  variates : 
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J  =  lim 


var  (X) 


k  ->  «,  (E(X)} 


k-1 

1  +  2  I 

j-l 


(1  - 


*>Pl 


=  1  +  2  I 
j  =  l 


Pj  . 


(2.21) 


and  is  proportional  to  the  initial  point  of  the  spectrum 
of  the  process   {X.}.   For  the  EAR(2)  process  this  is, 
using  (2.17)  , 


J  =  1  + 


2Y1Z1 


2y2z2 


(l-z1)  +  d-z2)  • 


(2.22) 


This  becomes  very  large  as  the  roots   z,   and   z»   approach 
1,  indicating  that  the  process  has  very  long  term  dependence 
in  it . 

Some  other  properties  of  the  EAR (2)  process  which  are 

of  interest  are  that  the  regression  of   X.   on   X.  ,   and   X.  „ 

^  l        l-l         i-2 


is  linear  in  the  given  values   x.  ,   and   x.  0   of   X.  , 

^  l-l         1-2        l-l 


and 


X 


i-2 


E(X.|Xi_1  =  x._i;  X._2  =  x._2) 


(l-a„)a,x.  ,  -f  a„x.  _  +  A  (l-a_ )  (1-a. +a_ )  , 
2   1  l-l     2  1-2         2      12 


(2.23) 


and  that  the  conditional  correlations  of   X.   and   X.  „ ,  given 

l         i-£   ^ 


xi-i-  • 


,  X.  .  , ,  are  zero  for   £  =  3,4, 

i-£-l 
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We  note  too  that  the  EAR (2)  model,  like  the  EAR(l)  model, 

is  slightly  degenerate  in  that  one  obtains  runs  of   X. 's  which 

are  fixed  multiples  of  the  previous   X.  ,   or   X.  „. 

l-l        i-2 

Joint  distributions,  higher-order  joint  moments  and  partial 

sums  of  the  {X.}   process  will  be  considered  elsewhere, 
l    r 


3.   THE  EXPONENTIAL  MOVING  AVERAGE  EMA(q)  MODEL 

The  EAR(l)  model  of  Gaver  and  Lewis  (1975-78)  led  to  the 
development  by  Lawrance  and  Lewis  (1977)  of  a  corresponding  first- 
order  exponential  moving  average  model;  this  took  the  form,  in 
the  backward  case,  of 


X.  = 

l 


3E.        w.p.   3 


(0  £6<_1;  i=  0,+l,+2,  ...)     (3.1 

BE.  +  E.  .   w.p.  (1-3) , 
i    l-l     c 


where  the   {E.}   is  again  a  sequence  of  i.i.d.  exponential ( A ) 

variables.   The   X.'s  have  an  exponential  marginal- distribution 

and  are  only  serially  dependent  for  lag  one;  this  model  is  highly 

tractable  and  a  full  account  of  the  statistically  useful  properties 

was  obtained.   The  forward  model  is  defined  as  a  random  mixture 

of   3E.   and   3E .  +  E.  ,  ,  instead  of   RE-   and   RE-  4-  E-  ■,  . 
i  l     i-i-l  i  l     l-l 

Lawrance  and  Lewis  (1977)  pointed  out  briefly  that  exten- 
sions to  second-order  moving-average  models  are  possible;  thus  we 
replace   E.  ,   in  (3.1)  by  another  EMA(l)  variable,  a  random 

linear  combination  of   3nE.  ,   and   3,  E .  -.    +  E.  „,  which  will  still 

1  l-l         1  l-l     i-2 

be  exponentially  distributed  and  independent  of  the   E.   variable. 
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Thus  the  second  order  backward  model,  EMA(2),  becomes 


32E. 


w.p.   S- 


X.  = 

l 


;2Ei  +  3lEi-l 


w.p.   (1-32)31 


(3.2) 


:2Ei  +  3lEi-l+  Ei-2   w,p-   (1-32)  (1"3i)  ' 


where   0  <  3  ,  3?  <_  1 ;  i  =  0,+l,+2,  ...  .   The  serial  dependency 
of  this  model  clearly  stops  at  the  second  lag  and 


pl  =  (1_62)  31U  "  (1-62)31};    p2  =  32(1  "  31)  (1  "  32)#   (3'3) 


This  model  reduces  to  the  independence  case  and  the  EMA(l)  model 
for  various  values  of   3-,   and   3?.   The  general  EMA(q)  model  takes 
the  form 


X.  = 

l 


3  E. 

q  i 


;  e.  +  3  ,e.  , 

q  l     q-1  l-l 


3  E.  +  3   ,E.  .  + 
q  l     q-1  i-l 

3  E.  +  3   ,E.  ,  + 
q  l     q-1  i-l 


+  3-.E.   ,, 
1  l-q+l 


w.p .   b 


q+1 


w.p .   b 


w.p .   b. 


+  3  E.   ,.  +  E.     w.p.   b, 
1  i-q+1     l-q  1 


(3.4) 


for   0  £  31#  32,  ...  ,  3   <  1;  i  =  0,  +1,  +2, 


and 


i  =  q+1  , 


b.  = 

l 


(1-0  ) 


(l-3i)3i_1   q  >  i  >_  2, 


(3.5) 


(1-3  ) 


•  (l-3i) 
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i  =  1  . 


Note  that  the   3-  's  can  be  obtained  uniquely  from  the   b. • s ;  there 
are  q+1   b.  's,  but  only  q   3's,  since  the  sum  of  the   b. 's  is  equal 
to  one.   It  is  simple  to  see  that  the   {X.}   have  exponential 
marginal  distributions.   The  serial  correlations  for  this  model 
clearly  have  the  cut-off  property  associated  with  moving  average 
schemes;  they  can  be  obtained  without  recourse  to  difference 
equations.   Premultiplication  of  (3.4)  by   X._    (r  >_  1)   and  then 
taking  an  expectation  gives 


El¥i-r'  =  Wl  +  •••  +  bl>  E(¥i-r' 

+  Vl<bq  +  ""  +  bl)  E(Ei-lXi-r» 
+  •  •  • 

+  B1(ba  +  b1)  E(E._q+1X._r)+blE(E..qX..r).     (3.6) 


This  simplifies  since  there  are  the  relations 

3i(bi+1  +  •••  +  b±)    =  bi+1      (q  >  i  >  1)  .  (3.7) 

Thus  on  converting  (3.6)  to  covariances  we  have 


:.  ,X.   )  =   Y   b  _,,    Cov(E.   ,X. 
l   l-r      Ln      q+l-m       l-m   l- 


Cov(X.,X.   )  =   )   b  ,,    Cov(E.   ,X.   )  .  (3.8) 

l   l-r      Ln      q+l-m       i-m'  l-r 
m=0   ^ 

The  covariances  on  the  right-hand  side  of  (3.8)  follow  from  (3.4)  as 

b    , ,  Var (E.   )     0  <  m  <  q 
q-m+1       i-m        —    —  ^ 

Cov(E.   ,X. )  =  <  (3.9) 

i-m   i     | 

otherwise . 
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Thus  (3.8)  becomes 


Cov(x.,Xi_r)    =      I     b  bq_m+r+1  Var(E..m+r)  (l<r<q)  (3.10) 


m=r 


with 


q-r+1 

[      b   b    ^  (1    <    r    <   q) 

L         v   v+r  —        —  ^ 


v=l 

0  (q+1    <    r    <    oo) 


p(c3}    =   Corr(X.,X.       )    =<  t3'11* 

r  l      l-r 


Thus  the  serial  correlations  are  just  lagged  products  of  the   bj_ 
sequence  and  the  formula  (3.11)  is  completely  analogous  to  the 
formula  for  the  serial  correlations  of  the  standard   MA(q)  process; 
see  Box  and  Jenkins,  1970,  p.  68.   It  can  be  seen  from  (3.11)  that 
all  the  correlations  are  nonnegative  and  it  may  further  be  shown 
that  they  are  bounded  above  by  1/4.   Note  too  that  since  the   .:  ^  's 
are  lagged  products  of  the   b.   sequence,  it  is  not  possible  to 
determine  the   b. 's  uniquely  from  the   p  g  's.   Therefore  it  is 
not  possible  to  uniquely  determine  the   3- 's  from  the   p  ^  's. 

Consider  now  the  index  of  dispersion   J,  defined  at  (2.21) . 
It  can  easily  be  shown,  from  (3.11),  that  for  the  EMA(q)  process, 

this  is  given  by 

^  <3+l 

J  -  2  -  J      b  (3.12) 

P  =  l 

This  is  maximized  when  all   b  s   are  equal  and  thus    6  =  1/(1+2.)  , 
I   =    1,2, ...,q.   These  values  of   3p   have  the  property  that  they 
give  equal  weights  to  the  q+1  possible  linear  combinations  which 
can  make  up  an   X.,  that  is   b.  =  l/(l+q),  i  =  1,2,...,  q+1. 
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The  maximum  values  of   J   are  then,  as   q   increases,  1.5,  1.666, 

1.750,  1.8000  and  generally,  1  +  q/(l+q)   with  2  as  limiting  value; 

thus  beyond  a  certain  point,  increasing  the  order  of  the  moving 

average  (which  can  conceptually  go  to  infinity),  has  little  effect. 

This  implies  that  the  over  all  dependence  in  the  process,  as 

expressed  by   J,  is  bounded   and  that  very  high  values  of   q   do 

not  substantially  increase  dependence. 

A  convenient  notation  for  the  EMA(q)  sequence   {X.}   is 

M.^  ,  meaning  that   X.   has  a  moving  average  structure  of  order  q 

over   E.,  E.  ,,  ...  ,  E.     using  the  parameters   3,3   ■,,...,$, 
l    l-l  l-q       3      r  q    q-1'        1 

In  this  notation,  M.g    can  be  expressed  in  terms  of   M.       by 
the  recursion 

M.(q)  =  3  E.  +  if^M.^:15  ,      q  =  1,2,...  (3.13) 

i      q  i    i   i-l 

where   {I.   }   is  an  independent  sequence  of  binary  variables 
taking  value  zero  with  probability   b   ,  =  3  • 


4.   THE  EXPONENTIAL  AUTOREGRESSIVE -MOVING  AVERAGE  PROCESSES 

EARMA(p,q) . 

4a .   Definitions 

We  have  defined  both  autoregressive  processes  and  moving- 
average  processes  in  exponential  variables  of  any  specified  orders, 
p   and   q.   Here  we  bring  them  together  into  a  single  process, 


18 


EARMA(p,q),  although  it  will  be  seen  that  the  method  of  combination 
is  not  unique.   We  will  then  have  a  process  of  great  flexibility 
in  modelling  dependent  exponential  variables,  bearing  favorable 
comparison  with  the  standard  ARMA(p,q)  process  in  modelling  dependent 
Gaussian  variables.   Jacobs  and  Lewis  (1977)  linked  the  two  first 
order  exponential  processes  EMA(l)  and  EAR(l),  giving  an  EARMA(1,1) 
mixed  model  and  obtained  the  serial  correlations,  some  higher 
order  explicit  results  and  discussed  central  limit  and  mixing 
properties.   For  the  general  mixed  process  we  shall  give  two  types 
of  model  but  restrict  ourselves  to  their  correlational  properties, 
and  in  particular  derive  the  difference  equations  satisfied  by 
the  serial  correlations;  these  are  similar  but  not  identical  to 
those  of  the  standard  ARMA  process.   The  special  process  EARMA(2,1) 
and  EARMA(1,2)  will  be  considered  in  more  detail. 

In  seeking  exponentially  distributed  mixed  autoregressive- 
moving  average  processes  we  will  work  from  the  pure  (backward) 

moving  average  process  EMA(q)  given  in  (3.4)  .   One  reason  why 
the  exponential  moving  average  and  exponential  autoregressive 
models  are  appealing  and  tractable  is  that  they  are  expressed 
in  terms  of  independent  exponential  variables.   If  this  property 
is  to  be  carried  over  into  the  mixed  models,  then  the  autoregressive 
contribution  should  enter  without  violating  this  feature;  thus, 
to  construct  the  EARMA(p,l)  process  we  replace  the   E._,  variable 
in  the  EMA(l)  of  (3.1)  by  &■_-,,    an  EAR(p)  variable.   This  is 
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independent  of   E.   in  (3.1)  because  it  is  a  function  only  of 

E.  ,  ,  E._9,  ...  .   The  defining  equation  for  the  EARMA(p,l)  process 

is  thus 


X.  = 

1 


BE. 

i 


BE.  +  AJP} 


w.p 


w.p.  1-6 


(0  <B<  1;  i=0,+l,+2, . . .) 


(4.1) 


which  is  the  model  treated  by  Jacobs  and  Lewis  (1977)  when 
p  =  1.   Similarly,  (3.4)  leads,  on  replacing   Ei_q   by   A^ 
to  the  EARMA(p,q)  process,  with  equation 


3  E. 

q  i 


l    E.  +  3   iE.  , 
q  l     q-1  l-l 


w 


.p.   b  ,, 

r     q+1 

w.p .   b 


X.  = 

l 


q  l     q-1  i-l         1  i-q+1 


(P) 


w.p.   b. 


3  E.  +  3   iE.  ,  +•••+  3..E.    .-  +  A.^'    w.p.   b, 
q  l     q-1  i-l         1  i-q+1     l-q  1 


(4.2 


for   i  =  0,  +1,  +2,...,  where  the   b^'s  are  defined  at  (3.5)  and 

(p.q) 


0  <  3X, 


,3   <  1.   Writing   X 
q  -  i 


as  a  variable  in  the 
EARMA(p,q)  process  based  on  the  moving  average  parameters 
3  ,  3  _-,  ,  ...  /  3-,  ,  the  mixed  process  can  be  defined  recursively  as 


x: 

i 


(P/q)  _ 


3  E. 

q  i 


3  E.  +  X.(p'q-1} 
q  l     i-l 


w.p 


6 


w.p.   l-l 


(i=0,+l,+2, . . . ) 


(4.3) 
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This  class  of  models  will  sometimes  be  written  as  EARMA  (p,q) 
to  signify  that  it  is  based  on  a  backward  moving  average. 

Consider  further  the  structure  of  the  mixed  model;  for  instance, 

X.   depends  on   E.  ,  E .  _,  ,     ...      and  not  on   E-,-i/  E-.9'  •••  > 
paralleling  the  standard  model  ARMA(p,q)  model.   In  contrast 
to  the  standard  model  it  is  also  possible  that  the  autoregressive 
aspect  could  be  absent  in  stretches  of  the  process  when  one  of 
the  pure  moving  average  selections  is  chosen  each  time.   Dependency 
would  still  be  retained  in  the  model  by  the  moving  average  part 
(apart  from  the   q  =  1   case  of  course) ;  while  this  is  not  at 
all  unnatural  there  can  be  other  situations  when  it  is  desired 
to  always  have  autoregressive  dependency.   Such  considerations 
lead  to  alternative  mixed  models;  initial  concern  at  the  non- 
uniqueness  of  these  models  is  best  allayed  by  realizing  that 
there  is  nothing  unique  about  the  standard  Gaussian  mixed 
ARMA(p,q)  models.   In  an  alternative  formulation  of  the  general 
mixed  model,  to  be  denoted  by  EARMA  (p,q),  the  shifted  form  of 
the  forward  moving  average,  briefly  mentioned  in  Section  3  is 
used.   The  retention  of  independence  between  successive  terms 
in  the  model  then  leads  to  the  EARMA  (p,l)  process  as 

'B-jAJPJ         w.p.  £1    , 

X.  =<  (i  =  0,+l,+2,  ...)  (4.4) 

1   fBlAi-l  +  Ei    W*p*   1_ei' 
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and  to  the  EARMA  (p,q)  as 


>  A.(p)  w.p.   b   ,  , 

q  l-q  r     q+1 

3  A.(p)  +  3   ,E.   ,.  w.p.   b  , 

q  l-q     q-1  i-q+1  q 

X.  =  /    I  (4.5) 

\ 

6  A.(p)  +  3   ,E.  „,,  +•••+  e,E.  -,  w.p.   b  , 

q  l-q     q-1  l-q+1         1  i-l  ^ 

3  A.(p)  +  3   nE.    ,.  +•••+  3-.E.  ,  +  E.      w.p.   b  , 
q  l-q     q-1  l-q+1         1  i_l     i  1 


for   i  =  0,  +1,  +2,  . . .  .   It  can  be  seen  from  (4.5)  that  this 
has  the  structure 

(Vi-q  W'P-   6q 

x{P'q)=  \         (P)      (q_i)  (i=0/±l,±2,...)  (4.6) 

1      13  a:p;  +  x:q  ;   w.p.   1-3 

I  q  l-q     i  *  q 

where   X.q      is  a  variable  in  the  shifted  forward  moving  average 

model  of  order  q-1  using  the  parameters   3   i  /  ...  ,  3-1  .   Thus 

the  autoregressive  dependence  is  always  present,  and  is  lagged 

q   values  in  arrears;  the  moving  average  variable  gives  greater 

flexibility  to  the  initial  form  of  the  dependence,  and  differs  for 

the  two  models.   In  EARMA  (p,q)  the  most  recent   E.   is  always 

included;  then  with  probability   (1-3  )3   -•   a  linear  combination 

of   E.   and   E.  ,   is  included,  and  so  on  moving  back;  thus 
l         l-l  ^ 

because  of  the  certain  addition  of  a  new   E.   each  time  there 

l 

cannot  be  runs  of  scaled  values;  further,  this  is  a  back  progression, 
natural  in  many  cases.   The  price  of  these  features  is  that  the 
model  can  exhibit  patches  of  independence  when  only  the   E.   is 
chosen,  i.e.  the  autoregressive  tail  is  not  chosen.   This  cannot 
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happen  in  the  EARMA  (p,q)  where  the  autoregres sive  dependency 
is  always  present;  however,  complicated  but  weak  runs  of  scaled 
values  are  just  possible  in  the  EARMA  (p,q)  model,  arising  from 
a  low  order  autoregressive  contribution  sucessively  being  chosen 
on  its  own.   Such  a  situation  would  be  extremely  rare.   Our  general 
feeling  is  that  in  practice  there  would  not  be  much  to  choose 
from  between  the  two  types;  a  third  type,  more  similar  to 
EARMA  (p,q)  than  the  other  can  be  formed  by  interchanging  the 
processes  in  (4.6)  with  a  suitable  shifting  of  scale.   This  is 
not  considered  here. 


4b .  Correlations  for  the  backward  mixed  model  EARMA (p,q) 
We  next  derive  equations  satisfied  by  the  serial 
correlations  of  the  EARMA  (p,q)  process,  denoted  here  simply  as 
EARMA(p,q).   Multiplying  each  side  of  the  defining  equations 

(4  2)  bv   X.     (r  ±    0)   and  taking  expectations,  gives 
l-r 


E(XiXi_r)  =  6q(bg+1  +  •••  +  bx)  E(EiXi_r)      r  =  +1,  +2,  ... 

+  3q_x(bg  +  ...  +  b,)  E(E._iXi_r) 

+  ..-  +  &1(b2  +  bx)  E(E._q+1Xi_r) 

+  b1E(A.(p)X.   )  (4.7) 

1    l-q  l-r 


This  equation  is  not  valid  for   r  =  0   since  the  expectation  of 
the  mixture  is  not  the  mixture  of  the  expectations  when  the 

2?. 


variables  are  identical.   Following  equations  (3.6),  (3.7)  and 
(3.8),  the  covariance  form  of  (4.7)  becomes 


Cov(Xi,Xi_r) 


q-1 


b  Ll    Cov(E.   ,X.   )  +  b.  Cov(A.(p),X.   )  . 
L  n      q+l-m      l-m   l-r     1       i-Q   i-r 
m=0   ^  ^ 


(4.8) 


It  now  becomes  easier  to  work  mainly  in  terms  of  correlations  and 
to  define 


p   =  Cov(X.,X.   )    and    K   =  Corr(E.,X.   )  (4.9) 

'r        l   l-r  r         i   l-r 


for   r  =  0 ,  +1 ,  + 2 ,  ...  .   Since  the   E.  and   X.  variables  have 
the  same  marginal  exponential  distribution,  (4.8)  becomes 


q-1 


(P) 


K     +  bn  Corr (A.^' ,  X. 
u~      q+l-m  r-m     1        i-q    i-r 
m=0   ^  ^ 


(4.10) 


for   r  =  +1,  +  2,  ...  .   To  calculate  the  cross-covariances 
between  the  autoregressive  and  mixed  process,  we  first  note 

that 

I   .  (d) 

w.p 


aA(P| 
1  l-l-q 


(P)_ 
li-q 


2  i-2-q 


p  i-p-q 


w.p 


w.p 


+  e 


i-q 


i  =  0,+l,+2, 


(4.11) 
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following  the  notation  at  (2.5)  and  (2.6).   The   r  . -term  in  (4.11) 
has  a  distribution  which  depends  only  on   E.,  such  as  was  determined 
for   p  =  2   in  Section  2.   Multiplication  of  (4.11)  by   X._   in 
order  to  calculate  correlations  leads  to 


Corr(A.(p)  ,X.   ) 
l-q   l-r 


=   7   a„anCorr(A.  .   X.   )  +  Cov(e.   ,X.   )/Var(E)  , 
j^,   £  I  i-£-q  l-r         i-q   l-r 


r  =  0,+l,+2,  ...  .  (4.12) 


We  now  wish  to  substitute  from  (4.10)  for  the  correlations  in  (4.12) 
and  so  obtain  a  difference  equation  for   p  .   However  we  do  not 
have  (4.10)  in  the  case   r  =  0.   Thus  in  (4.12)  when   i-r-(i--q)  =q 
this  substitution  is  not  possible,  that  is  when    £  =  r   if 
r  <_  p.   In  this  case 

Corr  (A.(p)  ,X.  )  =  b,  ,  (4.13) 

l-q   l      1 

as  may  be  seen  from  (4.2).   Thus  (4.10)  and  (4.12)  lead  to 


q-1 

y 

r     L _   q+l-m  r-m 
m=0   ^ 


P.     I      *>  +i.  K 


P*             q-1  2 

Y  a  a  {p       V   b  ,,   K   n  +  a  a  bn 

nL,  £  £   r-£    Ln      q+l-m  r-£-m  r  r  1 
£=1             m=0   ^ 


+  b  Cov(ei_  ,Xi_r)/Var(E)  (4.14) 
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2 

for   r  =  l,2,...,p;  if   r  >  p   then  the  term   a  a  b    is  omitted. 

The  asterisk  denotes  that  the    I    =   r   term  is  omitted  from  this 
summation  when   r  <_  p .   The  equation  simplifies  on  defining 

pQ  =  b^    and     Cr_g  =  Cov  (£i_q/^i_r) /Var (E)  ;  (4.15 


equation  (4.14)  may  then  be  written 

B  9;1  p* 

p      =      )       a.anp       „+./*>,,       {K  -        X      K-  .  .       }+   b,C         .       (4.16 

r         /,       I    lr- I  L~      q+l-m      r-m         0L,       r-£-m  1    r-q 

1=1  m=0      ^  1=1  M 

This  is  the  desired  general  result. 

Noting  that   K.  =  0   for   j  >_  1 ,  we  see  that  for   r  >  p+q, 
(4.16)  reduces  to  an  rth  order  difference  equation 

p   =  a,a,p   ,  +  a0a_p   „  +  •••  +  a  a  p        (r  >  p+q)  ,        (4.17 
r     1  lKr-l     2  2  r-2  p  p^r-p       —  *   ^   ' 

which  is  the  same  as  (2.16)  for  the  EAR(p)  process.   To  calculate 

the  initial   p+q-1   serial  correlations,  n.,  .  ...n     , ,  we  need 
r  -a  '  Ml      '  Mp+q-l 

KQ,  K_1,  ...  ,  K_        and   CQ,  C_lf  ...  ,  C_   1#-  explicit 
expressions  for  these  quantities  are  given  in  the  Appendix.   The 
correlation  structure  of  EARMA(p,q)  processes  is  thus  similar  to 
that  of  the  standard  ARMA(p,q)  processes;  the  only  difference  is 
that  initial  calculation  of  the  first   p+q   serial  correlations 
are  needed  to  start  the  difference  equation  (4.17),  rather  than 
the  first   p   as  in  the  ARMA(p,q)  case.   Note  finally  that  (4.16) 
is  only  strictly  true  when   p  >_  1 ,  q  >_  1 ,  although  similarities 
with  the  equations  for  the   (p  =  0,  q)  case  and   (p,  q  =  0)  case 
are  apparent . 
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These  results  all  apply  to  the  backward  model,  EARMA(p,q); 
for  the  forward  EARMA  (p,q)  model  (see  (4.5))  slightly 
different  correlation  equations  apply.   The  main  difference  is 
that  (4.8)  is  now 


(n)  9 

Corr(X.,X.   )  =  b   -  Corr(A.F',X.   )  +   Y   b   .   K       .   (4.18) 
l   l-r      q+1        i-g   i-r      _-.   q+l-m  r-q+m 


Using  (4.12)  we  then  find,  corresponding  to  (4.16),  that 

?  ?  ?* 

p   =   )   a.anp   „  +   )   b  , ,   {K       -   )   b   „     )+b  , ,r 

5=1       r-l  f;,   q+l-m   r-q+m    .£,   r-£-q+m      q+l^r-q 

(4.19) 


for   r  =  1,2,...   with   p   =  b 


,  ,  ;  as  before  explicit  calculation 
q+1  ^ 


of   Kn,  K  ,,...,  K     j0   and   C.f  C  ,,  ...  ,  C   ,.   are 
0    -1  -p-q+2         0    -1  -q+1 

required  to  obtain  the  first  p+q  serial  conditions.   Further 
comparisons  between  the  two  types  of  model  will  be  dealt  with 
elsewhere . 


5.   SPECIAL  CASES  OF  THE  EARMA (p,q)  PROCESS 

We  shall  give  the  explicit  versions  of  the  correlation 
equations  (4.16)  in  the  cases   (p  =  1,  q  =  1),  (p  =  2,  q  =  1) 
and   (p  =  1,  q  =  2);  these  are  considered  to  be  potentially  the 
most  useful. 
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(i)   EARMA(1,1) 

This  model  will  be  written  in  the  notation 


X 


w.p.   3 


BE.  +  A.  ,   w.p.   1-6 
1    l-l 


where 


A.  =  pA.  ,  +  £ . , 
l      l-l     l 


E  .   = 

1 


w.p 


E .   w.p.   1-p 


The  correlation  equations,  from  (4.16),  are  then 


pl  =  alalbl  +  b2Kl  +  blC0  '    pr  =  alalpr-l  '  (r  -  2) 


giving 


Pl  =  p(l-3)  +  (1-p)  3(1-3),    Pr  =  PPr_i,      (r  >  2 


This  agrees  with  the  result  (2.4)  of  Jacobs  and  Lewis  (1977). 

(ii)   EARMA(2,1) 

Using  (4.16)  to  obtain  the  results  for   p,   and   p    we  have 


pl  =  aiaibi  +  a?a2p-l  +  b2^Kl  ~  K-l^  +  blC0  ' 


Thus 
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(l-a2a2) p^ 

2  -1  -1 

=  a1a1(l-6)   -  a1a13(l-3)  (tTj^  +  tt2S   )  +  (i,+  S   i  )  3(1-6) 


and 

2 

P2  =  a1a1p1  +  a  a  (1-3) 


with 


pr  =  alalpr-l  +  a2a2pr-2  (r  >  3) 


(iii)  EARMA(1,2) 

In  this  case  (4.16)  gives 


and 


2 
p,  =  a,a,b,  +  b_K,  +   k2K0  +  ^ i  c_ i  '< 


P2  =  a1a1p1  +  b3(K2-K1)  +  b2(K1-KQ)  +  b^, 


p   =  a,  a,  p   ,        (r  >  3)  . 
r     1  1  r-1  — 


These  simplify  to 

p±   =    p(l-61)2(l  -  32)2  +  3132(l-32)  +  (tt1  +  S_1tt2)  3]_  (1-B]_)  U-B2)  2  , 
P2  =  PPl  -  $1B2(1-S2)  +  (tt1  +  S_1tt2)  (l-61)  32(1-B2)  , 


and 


p   =  an  a,  p   ,       (r  >  3 ) 
r     1  1  r-1  — 
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6 .   FURTHER  DEVELOPMENTS 

There  are  many  facets  and  properties  of  the  EARMA(p,q) 
process  which  will  be  investigated  in  later  papers.   Some  of  these 
properties  have  been  investigated  for  the  EAR(l)  process,  the 
EMA(l)  process  and  the  EARMA(1,1)  process  by  Gaver  and  Lewis 
(1975-78),  Lawrance  and  Lewis  (1977)  and  Jacobs  and  Lewis  (1977) 
respectively.   They  include  mixing  conditions,  infinite  divisibility, 
stationary  initial  conditions,  joint  distributions,  distributions 
of  sums,  spectra  and  higher  order  correlations.   It  should  also 
be  emphasized  that  all  the  serial  correlations  in  the  EARMA(p,q) 
process  are  positive;  this  aspect  of  the  process  can  be  broadened 
by  considering  antithetic  processes  and  will  be  discussed  else- 
where . 

Another  important  question  which  arises  is  whether  there 
are  non-normal  distributions  other  than  the  exponential  for  which 
mixed  autoregressive ,  moving  average  structures  can  be  defined 
analogous  to  EARMA(p,q) .   The  general  question  is  difficult  and 
has  been  considered  by  Gaver  and  Lewis  (1975-78)  for  the  first 
order-autoregressive  process.   However  it  is  clear  that  by  adding 

two  independent  EARMA(p,q)  processes,  say   {xf1^}   and  {X(2^}, 

i  i 

we  obtain  a  process  which  has  Gamma (2)  marginal  distributions, 

P{X.  <  X(1)  +  X(2)  <  x}  =   /   v  e"V  dv  , 
i  —   i       i    —        J 
1  0 
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and  ARMA(p,q)  correlation  structure.   Some  analysis  shows  that 
this  process  is  generated  directly  as  a  mixture  over  two  independent 
i.i.d.  exponential  (A)  sequences,  {E.  '}       and   {E,   ),  possibly 
scaled.   It  would  be  interesting  to  extend  this  process  to  the 
fractional  Gamma  case,  i.e.   k   not  an  integer  but  it  is  not 
clear  whether  this  process  exists  or  how  to  construct  it. 

For  the  first-order  autoregressive  case,  it  is  simple  to 
show  that  the  random  variable   e.   defined  at  (2.12)  is  infinitely 

divisible  and  therefore  that  the  solution  of  (2.4)  for   e.   exists 

1 

when   X.   is  Gamma (k, A)  and  has  a  Laplace-Stiel jes  transform 
which  is  just  (2.11)  raised  to  the  power   k, 


E(e    x)    =    (p  +  (1-p)  T—r)   ,        k  >  0,   A  >  0    (6.1) 

AtS 


However  it  is  difficult  to  invert  this  transform,  or  to  generate 
the  random  variable  on  a  computer  unless   «   is  an  integer. 

This  should  give  an  indication  of  some  of  the  interesting 
theoretical  questions  which  are  raised  by  the  EARMA  process. 
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APPENDIX:   Calculations  of   {K_  ,  r  =  0,1,2,  ...}   and 

{C_  ,  r  =   ,l,...,q-l}   for  EARMA(p,q)  models 


By  definition 


K    =  Corr(E.,X.^  )  =  Corr(E.   ,X.)   .         (A.l) 
-r  1   i+r  l-r   1 


By  the  usual  process  of  multiplication  and  expectation  it  is  found 
that 

Var(E.   )    0  <  r  <  q-1  , 


1"r  X  b,     Cov(E.   ,A.(P' 

I    1       i-r   i-c 


Cov(E.   ,X.)  =  v  ,  .  (A. 2) 


and  in  terms  of  correlations 


b     -  0  <  r  <  q-1  , 

q-r+1  —   —  ^ 

b1  Corr(Ei_r,A{^)   q  <_   r  < 


The  calculation  of  the  cross-correlations  between  the  independent 
exponential  sequence  and  the  derived  autoregressive  sequence 
proceeds  in  the  usual  way,  and  gives 


Corr  (E.  ,A.(p)  )  =  Cov  (E.  ,  e  .  ) /Var  (E)  =  d  ;                (A. 4) 

min(j,p)  () 

Corr(E.   .,A^pJ)  =     I              a„a„  Corr  (E .  .  ,A  )yJ  )    .                      (A. 5) 

±  —  J               1                     n   _-|              XX                  X  J       X    X, 
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We  only  need  (A. 5)  when   p  >_  2   and  for   j  =  l,2,...,p-2. 
Recursively  then  (A. 4)  and  (A. 5)  give 


Corr(E.   .,A.(p)) 

=  (a1a1)dQ  (j=l) 

2 
=  {  (a  a.^   +  (a2a2)>d0  (j=2) 

=  {(aia1)3  +  2(a1a1)  («2a2)  +  (a3a3)^  d0  (j=3)    (A. 6) 

4  2 

=  {  (a, a,)   +  3(oua,)   (a2a2)  +  2  (a,  a,  )  (a^a., ) 

2 

+  (a2a2)   +  (a4a4)}dQ   .  (j=4) 


Further  expressions  will  be  evident  based  on  the  fact  that  the 
sum  of  the  suffices  equals   j ,  that  all  possible  such  groups 
of  terms  are  present,  and  that  the  coefficient  of  a  particular 
term  is  the  number  of  distinct  orders  of  a  term  of  that  type. 
By  definition  we  have 

C_   =  Cov(e  .  ,X.+r)/Var  (X)  =  Cov  ( ei_r ,  X^  /Var  (X)  .  (A. 7) 

As  at  (A. 2)  above  we  have 

Cov(e.   ,X.)  =  b  ,,    Cov(e.   ,E.   )  ,   0  <  r  <  q-1 .       (A. 8) 
l-r   i      q+l-r       l-r   l-r        —   —  M 


Hence 
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C.r  =  Vl-r  d0   •  (A-9» 


The  calculation  of  (A. 8)  depends  on  the  form  of  the  error  random 

variable   £.   as  a  function  of  the  random  variable   E. .   For 
1  1 

example,  in  the  EAR (2)  case, 


d0  =  ^l  +  7T2//S  '  (A. 10) 
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